Summary of Regression Modeling Concepts
We use different probability models for different data types
- Binary outcomes: Bernoulli models
- Event rate outcomes: Poisson/Negative binomial models
- Time-to-event outcomes: Survival models
- Catch-all: Gaussian models
We use different link functions to connect these models with covariates
- Bernoulli models: logit link
- Count models: log link + offset
- Survival models: log link
- Gaussian models: identity link
Figure 1 sketches how the various models we have studied have analogous structures. To do: convert this sketch into a nicely formatted figure.
We use maximum likelihood estimation to fit models to data
- likelihood
- log-likelihood
- score function
- hessian
We use asymptotic normality of MLEs to quantify uncertainty about models
- observed information matrix
- expected information matrix
- standard error
- confidence intervals
- p-values
We use (log) likelihood ratios to compare models
Sometimes we adjust these comparisons for model size (AIC, BIC)